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Abstract 

Background: Current early childhood systems of care are not geared to respond to the complex needs of 
preschoolers at risk for mental health problems in a timely, coordinated, multidisciplinary, and comprehensive 
fashion. Evidence-informed policy represents an opportunity for implementing prevention, promotion, and early 
intervention at the population or at-risk level. Exposure to risk factors as well as the presence of clinical disorders 
can derail the developmental trajectories of preschoolers, and problems may persist if left untreated. One way to 
address these multiple research-to-policy gaps are systematic reviews sensitive to context and knowledge user 
needs, such as the realist review. The realist review is an iterative process between research teams and knowledge 
users to build mid-level program theories in order to understand which interventions work best for whom and 
under what context. 

Methods/Design: The realist review employs five 'iterative' steps: (1) clarify scope, (2) search for evidence, (3) 
appraise primary studies and extract data, (4) synthesize the evidence, and (5) disseminate, implement, and evaluate 
evidence, to answer two research questions: What interventions improve mental health outcomes for preschoolers 
at risk for socio-emotional difficulties and under what circumstances do they work? and what are the best models 
of care for integrating mental health interventions within pre-existing early childhood education (ECE) services for 
at-risk children? Knowledge users and researchers will work together through each stage of the review starting with 
refining the questions through to decisions regarding program theory building, data extraction, analysis, and design 
of a policy dissemination plan. The initial questions will guide preliminary literature reviews, but subsequent more 
focused searches will be informed by knowledge users familiar with local needs and further building of explanatory 
program theories. 

Discussion: Policy makers want to know what works best for whom, but are faced with a wide and disparate 
intervention literature for at-risk children. Applying evidence-based standards is a good start, but the chain of 
implementation between research results and how to match interventions sensitive to local context are ongoing 
challenges. 

Trial registration: Prospero registration number: CRD4201 4007301. 
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Background 

Current early childhood education (ECE) systems of care 
(providers and decision-makers) are not geared to re- 
spond to the complex needs of preschoolers and parents 
at high risk for serious mental health problems in a 
timely, coordinated, multidisciplinary, and comprehen- 
sive fashion. As the number of children in early day care 
has grown, both in overall numbers and in amount of 
time each child spends in early care, so too has the num- 
ber of children presenting in these settings with challen- 
ging socio-emotional difficulties [1]. Child care providers 
report that as many as 30% of their charges warrant 
special attention because of emotional regulation or be- 
havioral difficulties [2]. As a result, preschool children 
with these difficulties experience an expulsion rate from 
day care three times greater than the rate for school- 
aged children [3,4]. The gap is even greater for children 
outside of the day care system whose difficulties are not 
identified until kindergarten, when the behaviors are 
already entrenched. 

Who are these preschoolers and families at high risk? 
Can they be reliably identified so that they can receive 
appropriate support and intervention? In the ELDEQ 
study (Etude Longitudinale du Developpement des 
Enfants du Quebec), Japel and colleagues found that risk 
factors impacting child development included maternal 
depression, age and educational level of mothers, paren- 
tal substance abuse, family violence, child health, tem- 
perament, poverty, and neighborhood safety [5,6]. These 
risk factors were identifiable as early as 5 months of age 
in infants and were predictive of hyperactivity, inattention, 
aggression, anxiety, and oppositional behavior in kinder- 
garten. Poor mental health outcomes were dependent on 
the cumulative effect, i.e., the number of risk factors an 
infant experienced and the amount of time (years) a 
preschooler was exposed to the risk factors. 

Advances in assessment and treatment of socio- 
emotional functioning in infants and preschoolers as 
well as longitudinal studies in child development over 
the last 20 to 30 years have demonstrated that psycho- 
pathology (the combination of environmental and genetic 
risk factors) can be reliably identified in preschoolers and 
that many of these problems may be enduring if not 
treated early [7,8]. Infants and preschoolers can suffer 
mental health disorders such as anxiety and aggression or 
can manifest attachment difficulties and post-traumatic 
stress disorder if raised by a parent with their own risk fac- 
tors (mental illness, substance abuse, trauma) or in the 
context of family or intimate partner violence. Some 
children, struggling against many risk factors, will be 
thrown off of their normal developmental trajectory 
whereas other children, at genetic risk for psychiatric dis- 
orders, may show early manifestations of sub-syndromes 
or the full manifestation, with developmental variations, of 



psychopathology. Unfortunately, some children and their 
families are subjected to multiple environmental and 
genetic risk factors that interact/correlate (i.e., parental 
depression and adverse parenting) resulting in more 
negative outcomes. 

Therefore, preschooler psychopathology must be con- 
ceptualized as the manifestation of psychiatric symptoms 
and/or impaired development within the context of sensi- 
tive parenting or secure/insecure attachments. Early iden- 
tification and treatment become imperative to mitigate 
risk factors and increase protective factors, which in turn 
affect early brain development, caregiver/family interac- 
tions, and, ultimately, the developmental adaptations of 
growing children. 

In order to match intervention to need while optimizing 
development, various public health models have been pro- 
posed, such as the universal/selective/indicated preventive 
approach [9]. Preventive universal interventions are 
intended to support healthy development for all chil- 
dren, whereas selective approaches target children with 
high lifetime or imminent risk (e.g., Head Start pro- 
grams for poor families, home visitation). Indicated in- 
terventions target children already displaying minimal 
but detectable symptoms which may become major 
identifiable disorders. A fourth emerging level of inter- 
vention is for treating specific identifiable disorders in 
preschoolers and their families. Many new treatments 
are emerging and undergoing a first wave of evidential 
scrutiny at all levels of intervention. Some of these ther- 
apies are established treatments in older age groups that 
have been modified to account for developmental differ- 
ences (e.g., cognitive behavior therapy for younger chil- 
dren) whereas other interventions have been specifically 
designed to address risk factors specific to the early 
years (attachment-based therapies to target changing 
parent-infant interactions or parental sensitivity). While 
this schema has some intuitive appeal, the field in gen- 
eral is still struggling to match what interventions will 
mitigate which risk factors in order to produce the best 
developmental outcomes. 

Therefore, the case for intervening early and effect- 
ively is compelling; some studies have demonstrated 
that high-risk children and families benefit the most 
from high-quality intervention, delivered through either 
home or center-based care [10,11]. However, the evidence 
base is quite inconsistent and at times contradictory for 
comprehensive approaches because complex psychosocial 
interventions are quite context dependent [12,13]. Two 
separate reviews on US home visitation concluded that 
there were modest benefits at best from home visitation 
programs [14,15]. Cochrane reviews of home visitation in- 
volving disadvantaged mothers [16,17] and disadvantaged 
teenage mothers [18] found no difference in outcomes 
on parental or child risk factors. However, home-based 
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interventions delivered by trained professionals with 
high adherence to model fidelity (such as the Nurse- 
Family Partnership) show better and more consistent 
results compared to less rigorous programs [19]. A 
Cochrane review for center-based care found favorable 
outcomes in child IQ, behavior difficulties, and school 
achievement [20], but the review was criticized for 
insufficient control of confounding home visitation 
interventions [21]. 

What factors account for these inconsistent results? 
Home or center-based programs are quite varied and must 
be flexible to meet the needs of their target populations 
(i.e., the context in which service delivery takes place). 
Therefore, some programs are effective under some 
conditions (poverty, immigration), for some groups of 
children (developmental age), and for some risk factors 
(parental risks— depression, substance abuse; child risks- 
aggression, trauma; family risks— intimate partner vio- 
lence). Even the most rigorously tested programs with 
high fidelity (Nurse-Family Partnership) must be part of 
a comprehensive approach. For example, a home visitor 
may need to refer the child and family to a parenting 
program and needs to know which one is evidence 
based but also suits the client s needs (Incredible Years? 
Triple P? Parents as Teachers?). Any intervention pro- 
gram should be seen as an entry point into a system of 
care, rather than as a single model of care. Furthermore, 
overarching principles guiding evidence-informed pol- 
icymaking common to all early interventions are only 
now starting to emerge. For example, Astuto and Allen 
found that parental engagement was crucial for the suc- 
cess of any comprehensive intervention, whether it is 
home or center based [12]. At the higher policy level, 
different programs involved in early child development 
clash over ideological and historical differences on 
whether to focus on socio-emotional development or 
cognitive and literacy skills. National systems of care 
have differing priorities about the importance of the early 
years, affecting overall resource allocation for child welfare. 

Programming decisions based strictly upon the current 
available evidence may be misleading or fail to recognize 
the multiple needs of at-risk children, parents, and com- 
munities. Since risk factors for children with mental 
health challenges are nonspecific and quite context 
dependent, this knowledge gap needs to be synthesized 
for policy makers involved in program design of devel- 
opmentally sensitive services. Currently, no reviews, pol- 
icy documents, frameworks, or models exist to help 
decision-makers design new or review existing programs 
(resources, training, staffing, and coordinated services) 
around mental health interventions within child welfare, 
early education, or public health systems of care. 

Traditional systematic review methods which focus on 
measuring and reporting on effectiveness of interventions 



are not easily applied to interventions that take place in 
real world' health and social systems that are more com- 
plex and uncontrolled than research-based settings. Imple- 
mentation science brings an evaluation framework to 
specific patient populations with complex needs [22]. A 
further refinement of implementation science specifically 
designed to take into account how interventions may be 
modified by context is the realist review [13]. This type of 
review requires a broader examination of all relevant in- 
formation than would typically occur in a meta-analysis. It 
is an iterative process integrating and synthesizing know- 
ledge from the scientific literature with input from know- 
ledge users (KUs) familiar with local needs and barriers. 
KUs are actively involved with the research team to build 
mid-level program theories. 

The realist review approach is especially suitable for 
complex policy questions as it attempts to illuminate the 
mechanisms of how interventions work under different 
contexts. It makes explicit the context (C), mechanisms 
(M) of change, and outcomes (O) (CMO configurations) 
in order to identify what interventions work best, for 
whom, and under what circumstances. This is a particu- 
larly appropriate methodology for synthesizing complex, 
difficult to interpret evidence from multiple settings, i.e., 
merging home, community, and clinic-based interven- 
tions, and where more than one change mechanism or 
multiple change mechanisms acting simultaneously may 
be operating. 

This realist synthesis comes at a time when there is a 
critical need for integrated knowledge in the field of 
ECE concerning preschoolers at high risk for socio- 
emotional difficulties, an at-risk population and policy 
area heretofore neglected. Working with knowledge 
users responsible for ECE policy at the provincial level, 
this proposal provides a novel approach with a clear, prac- 
tical methodology for synthesizing knowledge relevant for 
policy recommendations. 

Methods/Design 

Our synthesis is a collaborative initiative between re- 
searchers and knowledge users across different policy, 
disciplinary, and organizational boundaries. The inter- 
pretative approach inherent in this review of complex 
early childhood interventions will benefit from the em- 
pirical perspective of researchers coupled with decision- 
makers and practitioners who work in the system and 
therefore are knowledgeable about the context. Know- 
ledge users and researchers will work together through 
each stage of the review process starting with refining 
the research questions through to decisions regarding 
data extraction, data analysis, and design of a dissemin- 
ation plan. The initial research questions will serve to 
guide preliminary literature reviews but can be modified 
by knowledge users in the course of the review if 
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deemed relevant: (1) What early childhood interventions 
improve mental health outcomes for preschoolers at risk 
for socio-emotional difficulties and under what circum- 
stances do they work? and (2) What are the best models 
of care for integrating mental health interventions within 
pre-existing ECE services for children at high risk for 
socio-emotional difficulties? 

The primary focus of a realist synthesis is to make ex- 
plicit the underlying theories about the causal mechan- 
ism of the complex intervention of interest [23]. This is 
accomplished by examining patterns of interaction be- 
tween the context, mechanism, and outcome in a sample 
of primary studies. Our goal is develop a realist program 
theory to explain how early childhood interventions 
work in different contexts to achieve improved mental 
health outcomes for preschool children. We will employ 
the five steps outlined in the realist template to guide 
our review: (1) clarify scope, (2) search for evidence, (3) 
appraise primary studies, (4) extract data, (5) synthesize 
the evidence, and (6) disseminate, implement, and evalu- 
ate [13]. Although the steps are described in a linear 
fashion, progression through the steps is an iterative 
process and it may be necessary to revisit earlier steps as 
the evidence begins to emerge. More specifically, in con- 
cert with knowledge users, we will: 

1. Clarify the scope of the review by searching for 

candidate (middle range) theories which explain how 
and why early childhood interventions work under 
different contexts. More than one middle range 
theory may explain the influence of context on a 
mechanism to produce an outcome. Therefore, an 
initial scoping search will be conducted to map the 
range of possible relevant theories for consideration 
and refine the research questions. We will extract 
the following data from all studies included in the 
scoping search: study characteristics (year, 
population, sample size, setting, and type of 
intervention or program), any explicit reference to a 
theory underlying the intervention, and any 
theoretical description linking context (C), 
mechanism (M) of action, and outcome (O). Data 
will be summarized in a table, and researchers and 
knowledge users will work together to review the 
results of the mapping exercise and begin to develop 
a program theory. Our beginning program theory 
will outline the contexts in which, populations for 
which, and main mechanisms by which important 
mental health outcomes for preschool children are 
expected to be achieved. Researcher and knowledge 
user consensus around the explanatory basis of these 
complex programs is vital to the success of the 
review process, and we anticipate that this work will 
initially consume a large portion of our project time. 



We will examine the evidence from the different 
intervention studies and negotiate with our 
knowledge users to adjudicate between the rival 
theories and identify which theory, or set of 
theories, might be important for inclusion in our 
program theory [24]. A realist review can generate a 
large number of paths to explore; therefore, we will 
use the following questions to guide refinement of 
our research objectives: Do the interventions work 
as predicted? Which theory/theories fit best? How 
do the different interventions work in different 
settings, for different groups? What are the policy 
implications for the different interventions? We will 
keep a project journal, separate from meeting notes, 
to document points of contention, key decision 
points, and insights arising from review of the 
evidence and researcher-knowledge user 
negotiations. 

2. Our purposive search for the evidence, focused on 
finding data that can be used to test our program 
theory, will require a combination of formal 
protocol-driven strategies and more informal 
approaches such as browsing and snowball methods 
(checking references of references and citation 
tracking). The search strategy will be guided by an 
information scientist and will evolve as the evidence 
emerges through each step of the review process. 
Databases to be searched include, but are not limited 
to, ClinicalTrials.gov; Cumulative Index to Nursing 
and Allied Health Literature (CINAHL); The 
Cochrane Library, including the Cochrane Central 
Register of Controlled Trials (CENTRAL); 
Educational Resources Information Centre (ERIC); 
EMBASE; Latin American and Caribbean Literature 
on the Health Sciences (LILACS); MEDLINE; 
ProQuest Nursing and Allied Health; PsycINFO; 
Web of Science; and WorldCat. 
Other sources: Due to the nature of our research 
questions, we anticipate that gray literature searches 
will yield important sources of information. Gray 
literature sources from Canada, Europe, and the 
USA will be searched, including but not limited to 
Canadian Health Research Collection, Canadian 
Public Policy Collection, Grey Literature Report, and 
OpenSIGLE. 

We will search the websites of organizations 
relevant to mental health and child health and the 
websites of federal and provincial government 
departments, including but not limited to Canadian 
Institutes of Health Research, Centre of Excellence 
for Early Childhood Development, CMA Infobase, 
Institute for Research on Public Policy, International 
Network for Early Child Development, International 
Organization for Early Intervention, National 
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Guidelines Clearinghouse, Offord Centre for 
Children at Risk, World Health Organization, and 
World Infant Mental Health Organization. Where 
applicable, table of contents of specific infant mental 
health journals will be hand searched, e.g., Infant 
Mental Health Journal, Child Development, Journal 
of Child Psychology and Psychiatry, Journal of 
American Academy of Child and Adolescent 
Psychiatry, Child Abuse and Neglect, and 
Developmental Psychopathology. Key policy experts 
will be identified and approached as well for 
literature, policy, or program information. 
Based on the goals of the study and the two research 
questions, the following inclusion! exclusion criteria 
will be used initially to inform the search process. 
After developing the CMO configurations, it is 
possible (consistent with a realist approach) that 
knowledge users and the research team may modify 
these criteria: 
Included: 

• Studies (qualitative or quantitative) focused on 
any type of complex intervention involving at-risk 
preschoolers. 

• Should address risk factors or adversity in child 
or parent and include at least one mental health 
outcome. 

• Should include reliable assessment of baseline 
mental health diagnosis or risk factors. 

• Studies focused on preschoolers at risk for 
socio-emotional difficulties (anxiety, depression, 
aggression). 

• Age range, 0 to 5. 

• Relevant reviews and meta-analyses. 

• Parenting group interventions including parents 
or children at high risk will be included (i.e., 
generic parenting groups such as groups for 
psychoeducation will be excluded). 

Excluded: 

• Studies examining autism, developmental, and 
communication disorders, as they are more 
related to child developmental-neurological 
factors. 

• Studies involving adults over the age of 19 that 
do not report data for the age group 0-5 years 
separately. 

• Studies focused on organic causes of intellectual 
or emotional impairment, i.e., fetal alcohol 
syndrome, or extreme prematurity. 

• Studies focusing primarily on assessment with no 
intervention component, no comparison group, 
and no outcome measures. 

Screening of studies will take place in two stages. 
First, two team members will independently screen 



all titles and abstracts for inclusion. Disagreements 
will be resolved by discussion or, if necessary, the 
senior investigator will adjudicate. In a second round 
of screening, the same two team members will 
independently examine the full text of included 
citations for suitability. 

3. Next, we will appraise included primary studies using 
forms and tools agreed to collaboratively by the 
knowledge user-research team. Selection and appraisal 
of documents in a realist review is carried out from a 
heuristic perspective to enrich the proposed CMO 
model. Given the diverse nature of likely sources of 
data, we will apply a 'fitness of purpose' approach to 
appraise primary sources [25] . Two team members 
working independently will assess each study for 
relevance (Does the study add anything to our 
program theory?) and rigor (Are the conclusions 
drawn by the researcher adequately supported to 
make a credible contribution to test our program 
theory?). A summary table outlining study authors, 
publication date, objectives, and type of study, 
conclusions, and appraisal data will be developed. 

4. Data extraction will be guided by the program 
theory outlined in step 1 and will evolve in an 
iterative manner to accommodate possible 
re-examination of studies as new codes emerge. One 
team member will conduct the first round of data 
extraction to examine the nature of the interventions 
reported in the primary sources. The basic 
components of a realist causal explanation include 
articulation of the context, mechanism of action, and 
outcomes of intervention of interest [26]. A matrix 
will be developed using Excel spreadsheets to 
organize extracted data into the basic context- 
mechanism-outcome configuration (step 1). In 
addition, we will extract any data related to relevant 
concepts or theories. As data are sorted, we will look 
for patterns (demi-regularities) to confirm or modify 
our program theory. 

We will track how different studies are used or 
omitted from this first round. A second round of 
data extraction will follow our second search and 
will be used to continue to build and refine our 
program theory. 

5. Data analysis and synthesis will be a joint effort of 
the researcher and knowledge user team and will be 
guided by the research questions. The primary goal 
of this task is refining our program theory 
improving our understanding of how interventions 
work in different contexts to achieve important 
mental health outcomes for preschool children. Each 
primary study will be examined according to how it 
supports, weakens, modifies, supplements, or 
refocuses our CMO configuration [26]. This critical 
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approach will assist with specifying the CMO 
elements and developing our program theory. 
6. The final step in the review process will involve 
reporting our findings, development of 
recommendations and a plan for dissemination. Our 
findings will be reported using the items listed in the 
RAMESES reporting standards for realist synthesis 
[27]. The knowledge users and researchers will work 
together to identify important stakeholder groups 
and develop recommendations taking into account 
the context and implementation chains of the 
different stakeholder groups. A brainstorming 
exercise will be completed using an online forum to 
identify potential barriers for the uptake of the 
recommendations. A dissemination plan will be 
designed to address these barriers and include 
successful strategies that have worked with the 
stakeholder groups in the past. 

Integral to the success of our projects work is the use 
of a knowledge broker familiar with local resources who 
will be an integral part of the dissemination process, 
merging policy, and academic perspectives. The know- 
ledge broker will assist with crafting recommendations 
that are sensitive to local policy context using the appro- 
priate lexicon. 

Discussion 

Many prospective and retrospective longitudinal studies 
of the risk factors that can affect developmental trajector- 
ies during a period of rapid brain growth have demon- 
strated that this sensitive period can be a tipping point 
leading to life-long health or disorder [28]. Within these 
trajectories, certain subgroups of children are more at risk 
due to exposure to multiple, cumulative, and prolonged 
environmental or genetic risk factors, such as poor attach- 
ment, maternal depression, and poverty. Infant mental 
health researchers have identified another subgroup of 
children with the onset of psychopathology (with or with- 
out environmental triggers), highlighting an early predis- 
position to partial or full psychiatric disorders such as 
disruptive behavior or anxiety disorders. These longitu- 
dinal studies highlight that it is no longer acceptable to 
adopt a wait and see' attitude coupled with the prevailing 
attitude that 'it is just behavior' and ignore the evidence 
that preschoolers can experience intense psychological 
distress with potential long-term sequelae. From the 
treatment perspective, 'not doing anything' is also a 
missed opportunity for preventive interventions within 
this window of rapid development. 

Policymakers want to know what works best and for 
whom, but are faced with a wide and disparate literature 
on intervention in the early years. Applying evidence- 
based standards to interventions is a good start, but the 



chain of implementation between research results and 
how to match interventions to local context are ongoing 
challenges. One way to address these multiple research- 
to-policy gaps is by using systematic review methodolo- 
gies sensitive to local context and knowledge user needs, 
such as the realist review. Our first research question is 
formulated to address this specific need around evidence- 
based interventions for preschoolers at high risk for socio- 
emotional challenges. In addition, systemic factors such as 
lack of coordination between departments mandated with 
child development, continuity of care across developmen- 
tal phases, simultaneous parallel services for children and 
parents (two generation models), and uneven specialized 
personnel training have hampered a consistent approach 
to intensive and continuous interventions for this popu- 
lation, the focus of our second research question. Work- 
ing with local knowledge users, the research team will 
synthesize evidence and contextual factors to shape pol- 
icy recommendations around embedding mental health 
services for at-risk preschoolers. 

Abbreviation 

KU: Knowledge user. 
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